Drawer-level immersion-cooling with hinged, liquid-cooled heat sink

ABSTRACT

Cooling apparatuses and methods of fabrication are provided which facilitate immersion-cooling of an electronic component(s). The cooling apparatus includes a drawer-level enclosure sized to reside within an electronics rack. The drawer-level enclosure includes a compartment which accommodates one or more electronic components to be cooled. A dielectric fluid is disposed within the compartment. The dielectric fluid includes a liquid dielectric which at least partially immerses the electronic component(s) within the compartment(s). A hinged, liquid-cooled heat sink is also disposed within the compartment of the enclosure. The heat sink operatively facilitates cooling the one or more electronic components via the dielectric fluid within the compartment, and is rotatable between an operational position overlying the electronic component(s), and a service position which allows access to the electronic component(s).

BACKGROUND

As is known, operating electronic components produce heat. This heatshould be removed in order to maintain device junction temperatureswithin desirable limits, with failure to remove heat effectivelyresulting in increased component temperatures, potentially leading tothermal runaway conditions. Several trends in the electronics industryhave combined to increase the importance of thermal management,including heat removal for electronic components, including technologieswhere thermal management has traditionally been less of a concern, suchas CMOS. In particular, the need for faster and more densely packedcircuits has had a direct impact on the importance of thermalmanagement. First, power dissipation, and therefore heat production,increases as device operating frequencies increase. Second, increasedoperating frequencies may be possible at lower device junctiontemperatures. Further, as more and more devices or components are packedonto a single chip, heat flux (Watts/cm²) increases, resulting in theneed to remove more power from a given size chip or module. These trendshave combined to create applications where it is no longer desirable toremove heat from modern devices solely by traditional air coolingmethods, such as by using air cooled heat sinks with heat pipes or vaporchambers. Such air cooling techniques are inherently limited in theirability to extract heat from an electronic component with high powerdensity.

The need to cool current and future high heat load, high heat fluxelectronic devices therefore mandates the development of aggressivethermal management techniques, using, for instance, liquid cooling.

SUMMARY

The shortcomings of the prior art are overcome and additional advantagesare provided through the provision, in one aspect, of a coolingapparatus which includes an enclosure, a dielectric fluid, and a hinged,liquid-cooled heat sink. The enclosure is sized to reside within anelectronics rack, and includes a compartment accommodating one or moreelectronic components to be cooled. The dielectric fluid resides withinthe compartment, and includes a liquid dielectric that at leastpartially immerses the one or more electronic components to be cooled.The hinged, liquid-cooled heat sink is disposed within the compartmentof the enclosure, and operatively facilitates cooling the one or moreelectronic components via the dielectric fluid within the compartment.The hinged, liquid-cooled heat sink is rotatable between an operationalposition overlying the one or more electronic components within thecompartment, and a service position which allows access to the one ormore electronic components within the compartment.

In another aspect, a cooled electronics rack is provided which includesan electronics rack comprising a plurality of electronic systems to becooled, and a cooling apparatus. The cooling apparatus includes: aplurality of drawer-level enclosures sized to reside within theelectronics rack, each drawer-level enclosure comprising a compartmentaccommodating a respective electronic system of the plurality ofelectronic systems; a dielectric fluid within each compartment, thedielectric fluid comprising a liquid dielectric at least partiallyimmersing the respective electronic system within the compartment; andat least one hinged, liquid-cooled heat sink associated with at leastone drawer-level enclosure of the plurality of drawer-level enclosures,each hinged, liquid-cooled heat sink being disposed within thecompartment of a respective drawer-level enclosure of the at least onedrawer-level enclosure, and operatively facilitating cooling of therespective electronic system via the dielectric fluid within thecompartment, each hinged, liquid-cooled heat sink being rotatablebetween an operational position overlying the respective electronicsystem within the compartment, and a service position which allowsaccess to the electronic system within the compartment.

In a further aspect, a method of fabricating a cooling apparatus isprovided to facilitate cooling one or more electronic components. Themethod includes: providing an enclosure sized to reside within anelectronics rack, the enclosure comprising a compartment accommodatingthe one or more electronic components; providing a dielectric fluidwithin the compartment, the dielectric fluid comprising a liquiddielectric at least partially immersing the one or more electroniccomponents; and hingedly mounting a liquid-cooled heat sink within thecompartment of the enclosure, the hinged, liquid-cooled heat sinkoperatively facilitating cooling of the one or more electroniccomponents via the dielectric fluid within the compartment, and thehinged, liquid-cooled heat sink being rotatable between an operationalposition overlying the one or more electronic components within thecompartment, and a service position which allows access to the one ormore electronic components within the compartment.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more aspects of the present invention are particularly pointedout and distinctly claimed as examples in the claims at the conclusionof the specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1A is an elevational view of one embodiment of a liquid-cooledelectronics rack with drawer-level immersion-cooling of electroniccomponents and/or systems, in accordance with one or more aspects of thepresent invention;

FIG. 1B is a cross-sectional elevational view of one immersion-cooledelectronic system of the liquid-cooled electronics rack of FIG. 1A, inaccordance with one or more aspects of the present invention;

FIG. 2A is a schematic depiction of another embodiment of aliquid-cooled electronics rack with drawer-level immersion-cooling ofelectronic components and/or systems thereof, and showing onedrawer-level enclosure coupled to a fill and drain tool for servicing,in accordance with one or more aspects of the present invention;

FIG. 2B is a schematic of one embodiment of a modular cooling unit for aliquid-cooled electronics rack such as depicted in FIG. 2A, inaccordance with one or more aspects of the present invention;

FIG. 2C depicts the liquid-cooled electronics rack of FIG. 2A, with onedrawer-level enclosure shown removed from the electronics rack, and theassociated liquid-cooled heat sink shown rotated to a service positionto allow access to one or more electronic components within thedrawer-level enclosure, in accordance with one or more aspects of thepresent invention;

FIG. 3A is a plan view of one embodiment of a drawer-level,immersion-cooled electronic system of a liquid-cooled electronics racksuch as depicted in FIGS. 2A-2C, with the hinged, liquid-cooled heatsink shown in operational position, in accordance with one or moreaspects of the present invention;

FIG. 3B is a cross-sectional elevational view of the drawer-level,immersion-cooled electronic system of FIG. 3A, taken along line 3B-3Bthereof, in accordance with one or more aspects of the presentinvention;

FIG. 3C is a cross-sectional elevational view of another embodiment of adrawer-level, immersion-cooled electronic system, wherein single-phase,natural convection immersion-cooling replaces the two-phaseimmersion-cooling of FIGS. 3A-3B, in accordance with one or more aspectsof the present invention; and

FIG. 4 is an elevational view of the drawer-level, cooled electronicssystem of FIGS. 3A-3C, illustrating rotation of the hinged,liquid-cooled heat sink between an operational position overlying theone or more electronic components being immersion-cooled, and a serviceposition which allows for access to the one or more electroniccomponents, in accordance with one or more aspects of the presentinvention.

DETAILED DESCRIPTION

Aspects of the present invention and certain features, advantages, anddetails thereof, are explained more fully below with reference to thenon-limiting embodiments illustrated in the accompanying drawings. Itshould be understood that the detailed description and examplesprovided, while indicating embodiments of the invention, are given byway of illustration only, and not by way of limitation. Varioussubstitutions, modifications, additions, and/or arrangements within thespirit and/or scope of the underlying inventive concepts will beapparent to those skilled in the art from this disclosure.

In a conventional raised-floor layout of an air-cooled data center,multiple electronics racks may be disposed in one or more rows. Asunderstood in the art, “electronics rack”, “rack”, “informationtechnology (IT) rack”, etc., may be used interchangeably, and unlessotherwise specified, include any housing, frame, support, structure,compartment, etc., having heat-generating components of a computersystem, electronics system, IT system, etc. A computer installation mayhouse several hundred or even several thousand microprocessors. Forinstance, in one or more implementations, a computer system may includemultiple rack-mounted servers, with the rack being enclosed in a cabinetcontaining multiple mounting slots called bays, each designed to hold adrawer in which is packaged the electronics hardware comprising, forinstance, a respective server node. One typical rack configurationutilizes multiple drawers stacked one above the other horizontally.

In one implementation, an electronics rack may be totally air-cooled. Insuch a case, one or more air-moving devices may be provided tofacilitate airflow through the electronics rack to cool theheat-dissipating electronic components or modules within the rack. Whiletraditionally, air-cooled electronics racks have been commonly used,they are limited in the amount of heat dissipation that can beaccomplished. This constraint poses a limit on the amount of computingperformance that can be achieved in an electronics rack usingair-cooling only.

To overcome this limitation, certain high-performance computer systemsnow utilize a combination of air-cooling and water-cooling. Forinstance, air-moving devices may still provide air-cooling tolower-power components, but water may be supplied to cold plates mountedto respective high-heat dissipating components, such as processormodules, to accommodate their higher-heat dissipation. Water may besupplied to and returned from each server node in parallel by supply andreturn manifolds at one side of the electronics rack. Althoughair-cooled and air- and water-cooled electronics racks are performingtheir functions at present power levels, continuing increases in powerconsumption to provide increased functional computing performancerequires ever-more effective cooling needs. In addition, theintroduction of 3-D chip packaging technology further exacerbates thecooling challenge.

Immersion-cooling is one possible solution to these issues. Inimmersion-cooling, the components to be cooled are immersed in adielectric fluid that dissipates heat, for instance, through boiling.The vapor is then condensed by a secondary liquid, using, for instance,finned condensers. Alternatively, single-phase natural convectionimmersion-cooling can be employed in combination with a liquid-cooledheat sink immersed within the dielectric fluid, as explained furtherbelow. Direct immersion-cooling of one or more electronic components ofan electronic system, such as a drawer-level electronic system of anelectronics rack, using dielectric fluid (e.g., a liquid dielectriccoolant) advantageously avoids the need for any forced air-cooling andenables total liquid-cooling of an electronics node, and thus, theelectronics rack within the data center. The use of dielectric fluidimmersion-cooling may offer several unique benefits over other coolingapproaches.

For example, the use of a dielectric fluid that condenses at atemperature above typical outdoor ambient air temperature would enabledata center cooling architectures which do not require energy intensiverefrigeration chillers. Also, the use of liquid immersion-cooling may,in certain cases, allow for greater compaction of electronic componentsat the electronic subsystem-level and/or electronic rack-level sinceconductive cooling structures might be eliminated. Unlike corrosionsensitive water-cooled systems, chemically inert dielectric coolant(employed with an immersion-cooling approach such as described herein)would not mandate copper as the primary thermally conductive wettedmetal. Lower cost and lower mass aluminum structures could replacecopper structures wherever thermally viable, and the mixed wetted metalassemblies would not be vulnerable to galvanic corrosion, such as in thecase of a water-based cooling approach. For at least these potentialbenefits, dielectric fluid immersion-cooling of one or more electronicsystems (or portions of one or more electronic systems) of anelectronics rack may offer significant energy efficiency and higherperformance cooling benefits, compared with currently available hybridair and indirect water cooled systems.

In the examples discussed below, the dielectric fluid may comprise anyone of a variety of commercially available dielectric coolants. Forexample, any of the Fluorinert™ or Novec™ fluids manufactured by 3MCorporation (e.g., FC-72, FC-86, HFE-7000, and HFE-7200) could beemployed. Alternatively, a mineral oil, such as SpecTrosyn Oil, offeredby Exxon-Mobil, may be employed if desired.

FIG. 1A is a schematic of one embodiment of a liquid-cooled electronicsrack, generally denoted 100, employing immersion-cooling of electroniccomponents and/or systems. As shown, liquid-cooled electronics rack 100includes an electronics rack 101 containing a plurality of electronicsystems 110 disposed, in the illustrated embodiment, horizontally so asto be stacked in drawers within the rack. By way of example, eachelectronic system 110 may be a server unit of a rack-mounted pluralityof server units. In addition, each electronic system may includemultiple electronic components to be cooled, which in one embodiment,may comprise multiple different types of electronic components havingdifferent heights and/or shapes within the electronic system.

The cooling apparatus is shown to include one or more modular coolingunits (MCU) 120 disposed, by way of example, in a lower portion ofelectronics rack 101. Each modular cooling unit 120 may be similar tothe modular cooling unit depicted in FIG. 2B, and described below. Themodular cooling unit includes, for example, a liquid-to-liquid heatexchanger for extracting heat from coolant flowing through a systemcoolant loop 130 of the cooling apparatus and dissipating heat within afacility coolant loop 119, comprising a facility coolant supply line 121and a facility coolant return line 122. As one example, facility coolantsupply and return lines 121, 122 couple modular cooling unit 120 to adata center facility coolant supply and return (not shown). Modularcooling unit 120 further includes an appropriately sized reservoir, pumpand optional filter for moving liquid-coolant under pressure throughsystem coolant loop 130. In one embodiment, system coolant loop 130includes a coolant supply manifold 131 and a coolant return manifold132, which are coupled to modular cooling unit 120 via, for example,flexible hoses. The flexible hoses would allow the supply and returnmanifolds to be mounted within, for example, a door of the electronicsrack hingedly mounted to the front or back of the electronics rack. Inone example, coolant supply manifold 131 and coolant return manifold 132each comprise an elongated rigid tube vertically mounted to theelectronics rack 101 or to a door of the electronics rack.

In the embodiment illustrated, coolant supply manifold 131 and coolantreturn manifold 132 are in fluid communication with respective coolantinlets 135 and coolant outlets 136 of individual sealed housings orenclosures 140 containing the electronic systems 110. Fluidcommunication between the manifolds and the sealed enclosures isestablished, for example, via appropriately sized, flexible hoses 133,134. In one embodiment, each coolant inlet 135 and coolant outlet 136 ofa sealed enclosure is coupled to a respective liquid-cooled vaporcondenser 150 disposed within the sealed enclosure 140. Heat removedfrom the electronic system 110 via the respective liquid-cooled vaporcondenser 150 is transferred from the system coolant via the coolantreturn manifold 132 and modular cooling unit 120 to facility coolantloop 119. In one example, coolant passing through system coolant loop130, and hence, coolant passing through the respective liquid-cooledvapor condensers 150 is water.

Note that, in general, fluidic coupling between the electronicsubsystems and coolant manifolds, as well as between the manifolds andthe modular cooling unit(s) can be established using suitable hoses,hose barb fittings and quick disconnect couplers. In the exampleillustrated, the vertically-oriented coolant supply and return manifolds131, 132 each include ports which facilitate fluid connection of therespective coolant inlets and outlets 135, 136 of the enclosures(containing the electronic subsystems) to the manifolds via the flexiblehoses 133, 134. Respective quick connect couplings may be employed tocouple the flexible hoses to the coolant inlets and coolant outlets ofthe sealed housings to allow for, for example, removal of a housing andelectronic subsystem from the electronics rack. The quick connectcouplings may be any one of various types of commercial availablecouplings, such as those available from Colder Products Co. of St. Paul,Minn., USA or Parker Hannifin of Cleveland, Ohio, USA.

One or more hermetically sealed electrical connectors 148 may also beprovided in each sealed enclosure 140, for example, at a back surfacethereof, for docking into a corresponding electrical plane of theelectronics rack in order to provide electrical and network connections149 to the electronic system disposed within the sealed enclosure whenthe electronic system is operatively positioned within the sealedenclosure and the sealed enclosure is operatively positioned within theelectronics rack.

As illustrated in FIG. 1B, in one or more implementations, electronicsystem 110 may comprise a plurality of electronic components 142, 143 ofdifferent height and/or type on a substrate 141, and is shown withinsealed enclosure 140 with the plurality of electronic components 142,143 immersed within a dielectric fluid 145. Sealed housing 140 isconfigured to at least partially surround and form a sealed compartmentabout the electronic system with the plurality of electronic components142, 143 disposed within the sealed compartment. In an operationalstate, dielectric fluid 145 pools in the liquid state at the bottom ofthe sealed compartment and is of sufficient volume to submerge theelectronic components 142, 143. The electronic components 142, 143dissipate varying amounts of power, which cause the dielectric fluid toboil, releasing dielectric fluid vapor, which rises to the upper portionof the sealed compartment of the housing.

The upper portion of sealed enclosure 140 is shown in FIG. 1B to includeliquid-cooled vapor condenser 150. Liquid-cooled vapor condenser 150 isa thermally conductive structure which (in one or more embodiments)includes a liquid-cooled base plate 152, and a plurality of thermallyconductive condenser fins 151 extending therefrom in the upper portionof the sealed compartment. A plenum structure 154 comprises part ofliquid-cooled base plate 152, and facilitates passage of system coolantthrough one or more channels in the liquid-cooled base plate 152. Inoperation, the dielectric fluid vapor contacts the cool surfaces of thethermally conductive condenser fins and condenses back to liquid phase,dropping downwards towards the bottom of the sealed compartment.

System coolant supplied to the coolant inlet of the housing passesthrough the liquid-cooled base plate of the liquid-cooled vaporcondenser and cools the solid material of the condenser such thatcondenser fin surfaces that are exposed within the sealed compartment tothe dielectric fluid vapor (or the dielectric fluid itself) are wellbelow saturation temperature of the vapor. Thus, vapor in contact withthe cooler condenser fin surfaces will reject heat to these surfaces andcondense back to liquid form. Based on operating conditions of theliquid-cooled vapor condenser 150, the condensed liquid may be close intemperature to the vapor temperature or could be sub-cooled to a muchlower temperature.

Advantageously, in immersion-cooling such as depicted in FIGS. 1A & 1B,all of the components to be cooled are immersed in the dielectric fluid.The system fluid can tolerate a larger temperature rise, whilemaintaining component temperatures, thus allowing a smaller flow rate,and higher inlet temperatures, improving energy efficiency of theresultant cooling apparatus.

Immersion-cooling of an electronic system, such as a server, may presentproblems with regards to servicing or replacing in the field one or moreof the components of the electronic system, such as one or more memorymodules. Servicing/replacing a component with an immersion-cooledelectronic system approach, requires that the electronic system bedrained, and that the sealed enclosure be opened to access theelectronic component(s) to be serviced or replaced. One embodiment ofthis is depicted in FIG. 2A.

Referring to FIG. 2A, another embodiment of a liquid-cooled electronicsrack, generally denoted 200, employing immersion-cooling of electroniccomponents or systems, is depicted, in accordance with one or moreaspects of the present invention. Cooled electronics rack 200 includesan electronics rack 201 containing a plurality of electronic systems 210disposed, in the illustrated embodiment, horizontally and stacked indrawers within the rack. By way of example, each electronic system 210may be a server unit of a rack-mounted plurality of server units. Inaddition, each electronic system may include one or more electroniccomponents to be cooled, which in one embodiment, may comprise multipledifferent types of electronic components having different heights and/orshapes within the electronic system.

The cooling apparatus is similar in certain respects to that depicted inFIGS. 1A & 1B. For instance, the cooling apparatus includes one or moremodular cooling units (MCUs) 220 disposed, by way of example, in a lowerportion of electronics rack 201. FIG. 2B illustrates one embodiment of amodular cooling unit 220.

Referring to FIG. 2B, modular cooling unit 220 includes, in one or moreembodiments, couplings to facility coolant loop 119, comprising facilitycoolant supply line 121 and facility coolant return line 122. Asillustrated, facility coolant is supplied through a control valve 222driven by a motor 223. Control valve 222 determines an amount offacility coolant to be passed through a liquid-to-liquid heat exchanger221, with a portion of the facility coolant possibly being returneddirectly via a bypass orifice 224. The modular cooling unit furtherincludes a system coolant loop having a reservoir tank 225 from whichsystem coolant is pumped, either by pump 226 or pump 227, intoliquid-to-liquid heat exchanger 221 for conditioning and output thereof,as cooled system coolant, to the liquid-cooled heat sinks within theelectronics rack being cooled. As illustrated, the cooled system coolantmay be supplied to the system coolant supply manifold 231, and returnedfrom the system coolant return manifold 232.

As illustrated in FIG. 2A, in one or more examples, flexiblemanifold-level supply and return hoses 228, 229 couple in fluidcommunication the respective modular cooling units 220 to coolant supplymanifold 231 and coolant return manifold 232. By way of example, coolantsupply manifold 231 and coolant return manifold 232 may each comprise anelongated rigid tube vertically mounted to electronic rack 201, or to adoor of the electronics rack.

In the embodiment of FIG. 2A, coolant supply manifold 231 and coolantreturn manifold 232 are coupled to respective liquid supply and returnconnectors associated with the individual enclosures 240 containing theelectronic systems 210. By way of example, enclosures 240 may bedrawer-level enclosures sized to reside within electronics rack 201 andconfigured to be removable from the electronics rack for, for instance,servicing of the electronic components or systems within the enclosure.Further, enclosures 240 are, in one or more embodiments, sealedenclosures, which may include an enclosure cover sealed in a fluid typemanner to enclosure 240 to form a fluid-tight compartment within theenclosure. Fluid communication between coolant supply and returnmanifolds 231, 232 and enclosures 240 containing the electronic systems210 can be established, for instance, via appropriately sized flexiblehoses 233, 234.

In one or more embodiments, each liquid supply and return connector ofan enclosure 240 is coupled to an inlet or outlet of a respectivehinged, liquid-cooled heat sink 250 (FIG. 2C) disposed within theenclosure 240. Heat removed from electronic system 210 via therespective hinged, liquid-cooled heat sink 250 is transferred from thesystem coolant via the coolant return manifold 232 and modular coolingunit 220 to facility coolant loop (FIG. 2B). In one example, coolantpassing through the system coolant loop, and hence, coolant passingthrough the respective hinged, liquid-cooled heat sink 250 compriseswater.

As with the embodiment of FIGS. 1A & 1B, fluid coupling between theelectronic systems or subsystems and coolant manifolds, as well asbetween the manifolds and the modular cooling unit(s) can be establishedusing suitable hoses, host barb fittings, and quick connect couplers. Inone example, the vertically-oriented coolant supply and return manifolds231, 232 each include ports 235, 236 which facilitate fluid connectionof the respective liquid supply and return connectors of the enclosures240 to the manifolds via the flexible hoses 233, 234. Respective quickconnect couplings may be employed to couple the flexible hoses to theliquid supply and return connectors of the enclosures and/or themanifolds to allow for, for example, removable of an enclosure andelectronic system from the electronics rack. The quick connect couplingsmay be any of the various types of commercially available couplings,such as those available from Colder Products Co., of St. Paul Minn.,USA, or Parker Hannifin, of Cleveland, Ohio, USA.

As in the embodiment described above in connection with FIGS. 1A & 1B,one or more hermetically sealed electrical connectors may be provided inassociation with each enclosure 240, for example, at a back surfacethereof, for docking into a respective electrical plane of theelectronics rack in order to provide electrical and network connectionsto the electronic system or components disposed within the enclosure 240when the electronic system is operatively position within the enclosureand the enclosure is operatively positioned within the electronics rack.

As explained further below in connection with FIGS. 3A-3C, a dielectricfluid, comprising (in one or more embodiments) a liquid dielectric, isdisposed within each enclosure for drawer-level immersion cooling of theone or more electronic components or system within the enclosure. Theuse of direct immersion cooling with dielectric coolant offers a meansfor providing enhanced cooling to the components and/or system.Advantageously, as described herein, field serviceability is alsomaintained, notwithstanding the use of drawer-level immersion cooling ofelectronic components, or system. One embodiment of the coolingapparatus illustrating field serviceability is depicted in FIGS. 2A &2C.

Referring first to FIG. 2A, flexible hoses 233, 234 of a selectedenclosure 240 are shown disconnected from the respective manifolds 231,232. Additionally, a fill/drain line 207 has been coupled to afill/drain port 241 of the enclosure 240 to couple in fluidcommunication a fill/drain cart 202 with the compartment containing thedielectric fluid within enclosure 240.

As illustrated, in on or more implementations, fill/drain cart 202includes a dielectric coolant storage tank 203 filled, at leastpartially, with a liquid dielectric 204. Additionally, a pump 205 and afilter 206 are housed within fill/drain cart 202, as well as tubing withsolenoid valves A, B, C & D. In the legend in FIG. 2A, the open ‘O’ andclosed ‘C’ states for the solenoid valves are provided for both a drainoperation, and a fill operation, where the drain operation drains liquiddielectric from the compartment of the enclosure, and the fill operationpumps liquid dielectric into the compartment of the enclosure.

In FIG. 2C, the enclosure 240 containing the electronic system to beserviced is shown removed, at least in part, from electronics rack 201.Once enclosure 240 has been pulled out, the enclosure cover (not shown)may be removed and the liquid dielectric may be drained into fill/draincart 202. To drain enclosure 240, solenoid valves A & D are opened, asshown in the table in FIG. 2A, while valves B & C remain closed.Although liquid dielectric will drain by gravity from enclosure 240,pump 205 of fill/drain cart 202 may be activated to expedite theprocess.

Once drained, servicing of one or more of the electronic components orelectronic system within the enclosure is facilitated by rotatinghinged, liquid-cooled heat sink 250 from an operational positionoverlying the one or more electronic components or system within thecompartment, to a service position (as illustrated in FIG. 2C), whichallows for access to the one or more electronic components or systemwithin the compartment. Embodiments of the hinged, liquid-cooled heatsink and mounting of the heat sink within the enclosure are describedfurther below with reference to FIGS. 3A-4.

After servicing the electronic components or system, solenoid valves A &D of fill/drain cart 202 may be closed and valves B & C opened. The pump205 may then be activated to pump liquid dielectric from dielectriccoolant storage tank 203 through particulate filter 206 and fill/drainline 207 to the enclosure 240. Filter 206 may be provided to ensure thatclean liquid dielectric is delivered back into the enclosure. Once theliquid dielectric reaches a desired level within the compartment ofenclosure 240, the pump may be turned off, and the fill/drain line 207may be disconnected. In one or more implementations, fill/drain port 241may comprise a quick disconnect coupling which facilitates connectionand disconnection of fill/drain line 207 to/from enclosure 240. Oncefill/drain line 207 is disconnected, the hinged, liquid-cooled heat sink250 may be rotated back into position; that is, returned to anoperational position overlying the one or more electronic components orsystem within the compartment of the enclosure. The enclosure cover (notshown) may then be refastened to the enclosure, and the enclosurereturned to an operational position within electronics rack 201. Systemcoolant flow can be re-established into the enclosure 240 by recouplingflexible hoses 233, 234 to the respective coolant supply and returnmanifolds 231, 232. Power may then be restored to the one or moreelectronic components/systems within the compartment of the enclosurethat was serviced. Note that this servicing may occur concurrent withcontinued operation of the other components/systems within theelectronics rack. In particular, power and coolant flow to only theselected node-level enclosure being serviced is interrupted during thefield servicing.

FIGS. 3A-3C depict in greater detail one embodiment of a coolingapparatus comprising a drawer-level enclosure 240. Referring first toFIGS. 3A & 3B, the electronic system 210, and in particular, thedrawer-level, immersion-cooled electronic system comprises, in one ormore implementations, a circuit board 211 supporting one or moreelectronic components or modules 212. As illustrated in FIG. 3B,electronic system 210 resides within a compartment 245 defined byenclosure 240 and cover 246. In a depicted implementation, multiplecover latches 244 are provided about the periphery of enclosure 240, aswell as a seal 247, which together secure cover 246 in a fluid-tightmanner to enclosure 240 and facilitate defining a fluid-tightcompartment 245. Within compartment 245 dielectric fluid is providedwhich includes, for instance, both liquid dielectric 260 and vapordielectric 261 (FIG. 3B) generated by pool boiling of the liquiddielectric 260. In the embodiment illustrated, electronic system 210 isfully immersed within liquid dielectric 260, by way of example only.

In the implementation of FIGS. 3A & 3B, the upper region of compartment245 includes hinged, liquid-cooled heat sink 250. Note that thisposition is provided by way of example only. Hinged, liquid-cooled heatsink may be implemented in a variety of configurations. For instance, asillustrated in FIG. 3A, the heat sink may substantially cover or overliethe entire electronic system 210 disposed in a lower region ofcompartment 245, or alternatively, could cover only a portion of theelectronic system, as required for a particular application. Hinged,liquid-cooled heat sink 250 is a thermally conductive structure which(in one or more embodiments) includes one or more thermally conductive,coolant-carrying tubes 252, and the plurality of thermally conductivefins 251 coupled to thermally conductive tube 252. In the two-phaseimplementation of FIGS. 3A & 3B, the plurality of thermally conductivefins 251 are a plurality of thermally conductive condenser fins. Inoperation, dielectric fluid vapor 261 rises and contacts the coolsurfaces of the thermally conductive condenser fins, where the vaporcondenses back to liquid phase, and drops downward towards the lowerregion of the sealed compartment 245.

FIG. 3C depicts an alternate embodiment, wherein single phase, naturalconvection cooling occurs. In this embodiment, the liquid dielectric 260substantially fills compartment 245, so that hinged, liquid-cooled heatsink 250 is immersed within liquid dielectric 260. Depending upon theheat flux level from the electronic components or system 210, theindividual heat dissipating components within the compartment may becooled by either the pool boiling approach of FIG. 3B, or the naturalconvection approach of FIG. 3C.

Note also, that the particular liquid-cooled heat sink 250 configurationillustrated is presented by way of example only, not by way oflimitation. For instance, other heat sink configurations could beemployed to facilitate condensing of dielectric fluid vapor and/ordirect cooling of liquid dielectric. For example, the hinged,liquid-cooled heat sink 250 could comprise a liquid-cooled base plate(not shown) with a plurality of thermally conductive condenser finsextending from the base plate in the upper region of the sealedcompartment. The liquid-cooled base plate could comprise any desiredcoolant-carrying channel configuration that facilitates passage ofsystem coolant through the liquid-cooled heat sink.

As illustrated in the embodiment of FIG. 3A, enclosure 240 may include aliquid supply connector 242 and a liquid return connector 243 associatedwith enclosure 240, and in particular, with a sidewall of the enclosure.Additionally, a flexible liquid supply hose 255 and a flexible liquidreturn hose 256 are provided within compartment 245 to couple in fluidcommunication the liquid supply and return connectors 242, 243,respectively, to the hinged, liquid-cooled heat sink 250. In one or moreimplementations, appropriate hose barb fittings and clamps may be usedto couple the flexible liquid supply and return hoses 255, 256 betweenthe connectors and heat sink. Further, in one or more embodiments,fill/drain port 241, as well as liquid supply and return connectors 242,243 are disposed in a lower region of the compartment 245 along, forinstance, a front side wall of enclosure 240 accessible, for instance,at the front of the electronics rack to which the enclosure is to beoperatively docked.

In operation, system coolant supplied to liquid supply connector 242 ofenclosure 240 passes through flexible liquid supply hose 255 to hinged,liquid-cooled heat sink 250 and cools the solid material of the heatsink such that, in the example of FIG. 3B, the thermally conductivecondenser fins that are exposed within the compartment to the dielectricfluid vapor (or to the liquid dielectric in the example of FIG. 3C) arewell below saturation temperature of the vapor. Thus, vapor in contactwith the cooler condenser fin surfaces will reject heat to the surfacesand condense back to liquid form. Based on operating conditions of thehinged, liquid-cooled heat sink 250, the condensed liquid may be closein temperature to the vapor temperature, or could be sub-cooled to amuch lower temperature.

In one or more enclosures 240, a liquid-cooled heat exchanger 250 may behingedly mounted to the enclosure via one or more appropriate hinges 253and brackets 254. For instance, liquid-cooled heat sink 250 may behingedly mounted to a sidewall of an enclosure 240 in an upper region ofcompartment 245, as illustrated in FIGS. 3B-4. In one or moreimplementations, the hinged, liquid-cooled heat sink 250 is orientedhorizontally within compartment 245 in the upper region of thecompartment when in operational position, overlying, for instance, theone or more electronic components or system being immersion-cooled.

As illustrated in FIG. 4, hinged, liquid-cooled heat sink 250 isrotatable from the operational position to a service position, where theheat sink extends substantially vertically from enclosure 240. Note thatin FIG. 4, the drawer-level enclosure 240 and electronic system 210sub-assembly is shown without liquid dielectric within compartment 245.In the service position, an operator is allowed access to the one ormore electronic components or system within compartment 245 to, forinstance, replace or repair the component/system. Note also that, in oneor more implementations, the flexible liquid supply and return hoses255, 256 remain connected and are of sufficient length and flexibilityto allow for rotation of the heat sink between the operational positionand service position without disconnection. Further, note that, in theexample depicted, flexible liquid supply hose 255 has a longer lengththan flexible liquid return hose 256, by way of example only. In one ormore implementations, the liquid supply and return connectors 242, 243(FIG. 3A) are located near the hinge 253 to allow for minimizing thelength of flexible liquid supply and return hoses 255, 256.

Note from the above discussion that a cooling apparatus and cooledelectronic system are provided herein which allow for fieldreplicability of one or more immersion-cooled electronic components orsystems within, for instance, a drawer of a multi-drawer electronicsrack to allow for servicing of the component/system, while the remainderof the electronics rack is powered and functioning, and continuing to becooled.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain theprinciples of one or more aspects of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand one or more aspects of the invention for various embodimentswith various modifications as are suited to the particular usecontemplated.

What is claimed is:
 1. A cooling apparatus comprising: an enclosuresized to reside within an electronics rack, the enclosure comprising acompartment accommodating one or more electronic components; adielectric fluid within the compartment, the dielectric fluid comprisinga liquid dielectric at least partially immersing the one or moreelectronic components; and a hinged, liquid-cooled heat sink disposedwithin the compartment of the enclosure and operatively facilitatingcooling the one or more electronic components via the dielectric fluidwithin the compartment, the hinged, liquid-cooled heat sink beingrotatable between an operational position overlying the one or moreelectronic components within the compartment, and a service positionwhich allows access to the one or more electronic components within thecompartment.
 2. The cooling apparatus of claim 1, further comprising aliquid supply connector and a liquid return connector associated withthe enclosure, and a flexible liquid supply hose and a flexible liquidreturn hose disposed within the compartment of the enclosure, theflexible liquid supply and return hoses respectively coupling in fluidcommunication the liquid supply connector and the liquid returnconnector to the hinged, liquid-cooled heat sink.
 3. The coolingapparatus of claim 2, wherein the flexible liquid supply hose andflexible liquid return hose each have a respective length which allowsfor rotating of the hinged, liquid-cooled heat sink between theoperational position and the service position while remaining connectedto the respective liquid supply and return connectors.
 4. The coolingapparatus of claim 3, wherein one of the flexible liquid supply hose orthe flexible liquid return hose is longer than the other.
 5. The coolingapparatus of claim 3, wherein the hinged, liquid-cooled heat sink ishorizontally-oriented and disposed in an upper region of the compartmentwhen in the operational position, and is vertically-oriented andextending away from the compartment when in the service position.
 6. Thecooling apparatus of claim 5, wherein the liquid supply connector andliquid return connector are disposed at a side wall of the enclosure. 7.The cooling apparatus of claim 1, wherein the enclosure comprises acover, the cover being removable to allow for transitioning of thehinged, liquid-cooled heat sink from the operational position to theservice position.
 8. The cooling apparatus of claim 1, wherein thehinged, liquid-cooled heat sink comprises at least one vapor-condensingsurface, and the dielectric fluid provides two-phase cooling of the oneor more electronic components within the compartment, the hinged,liquid-cooled heat sink with the at least one vapor-condensing surfaceresiding in an upper region of the compartment of the enclosure, theupper region of the compartment comprising a vapor region of thedielectric fluid.
 9. The cooling apparatus of claim 1, wherein thehinged, liquid-cooled heat sink is immersed within liquid dielectricwithin the compartment, the liquid dielectric providing single-phase,natural convection, immersion-cooling of the one or more electroniccomponents within the compartment.
 10. The cooling apparatus of claim 9,wherein the liquid dielectric fills a majority of the compartment, andthe hinged, liquid-cooled heat sink resides in an upper region of thecompartment of the enclosure.
 11. A cooled electronics rack comprising:an electronics rack comprising a plurality of electronic systems; and acooling apparatus comprising: a plurality of drawer-level enclosuressized to reside within the electronics rack, each drawer-level enclosurecomprising a compartment accommodating a respective electronic system ofthe plurality of electronic systems; a dielectric fluid within eachcompartment, the dielectric fluid comprising a liquid dielectric atleast partially immersing the respective electronic system within thecompartment; and at least one hinged, liquid-cooled heat sink associatedwith at least one drawer-level enclosure of the plurality ofdrawer-level enclosures, each hinged, liquid-cooled heat sink beingdisposed within the compartment of a respective drawer-level enclosureof the at least one drawer-level enclosure and operatively facilitatingcooling the respective electronic system via the dielectric fluid withinthe compartment, each hinged, liquid-cooled heat sink being rotatablebetween an operational position overlying the respective electronicsystem within the compartment, and a service position which allowsaccess to the electronic system within the compartment.
 12. The cooledelectronics rack of claim 11, wherein the cooling apparatus comprises aplurality of hinged, liquid-cooled heat sinks, the at least one hinged,liquid-cooled heat sink being at least one hinged, liquid-cooled heatsink of the plurality of hinged, liquid-cooled heat sinks, and eachhinged, liquid-cooled heat sink being associated with a respectivedrawer-level enclosure of the plurality of drawer-level enclosures, andbeing rotatable between the operational position overlying therespective electronic system within the compartment, and the serviceposition which allows access to the respective electronic system withinthe compartment.
 13. The cooled electronics rack of claim 12, furthercomprising respective liquid supply and return connectors associatedwith each drawer-level enclosure, and wherein the cooling apparatusfurther comprises flexible liquid supply hoses and flexible liquidreturn hoses disposed within the compartments of the plurality ofdrawer-level enclosures, wherein within the compartment of eachdrawer-level enclosure, a respective flexible liquid supply hose andrespective flexible liquid return hose couple in fluid communication theassociated hinged, liquid-cooled heat sink to the respective liquidsupply and return connectors associated with that drawer-levelenclosure.
 14. The cooled electronics rack of claim 13, wherein theflexible liquid supply hose and flexible liquid return hose within thecompartment of each drawer-level enclosure of the plurality ofdrawer-level enclosures has a respective length which allows forrotating of the associated hinged, liquid-cooled heat sink between theoperational position and the service position while remaining connectedto the respective liquid supply and return connectors.
 15. The cooledelectronics rack of claim 14, wherein each hinged, liquid-cooled heatsink is horizontally oriented and disposed in a upper region of theassociated compartment when in the operational position, and isvertically oriented and extending away from the compartment when in theservice position.
 16. The cooled electronics rack of claim 11, whereinthe at least one hinged, liquid-cooled heat sink comprises at least onevapor-condensing surface, and the dielectric fluid provides two phasecooling of the respective electronic system within the compartment ofthe respective drawer-level enclosure, each at least one hinged,liquid-cooled heat sink with the at least one vapor-condensing surfaceresiding in an upper region of the compartment of the respectivedrawer-level enclosure of the at least one drawer-level enclosure, theupper region of the compartment comprising a vapor region of thedielectric fluid.
 17. The cooled electronics rack of claim 11, whereineach at least one hinged, liquid-cooled heat sink is immersed withinliquid dielectric within the compartment of the respective drawer-levelenclosure of the at least one drawer-level enclosure, the liquiddielectric providing single phase, natural convection, immersing coolingof the respective electronic system within the compartment.
 18. Thecooled electronics rack of claim 17, wherein the liquid dielectric fillsa majority of the compartment, and each at least one hinged,liquid-cooled heat sink resides in an upper region of the respectivecompartment.